Trees

Overview

• What do we want to predict?
  • Return?
  • Return minus average return?
  • Rank of return (1, 2, 3, …)
  • Whether return will be in top, middle, bottom? Or deciles?
• What predictors do we want to use?
• What model do we want to use?

Some models

• Forests
  • Random forests
  • Boosting
• Neural networks
• Penalized regression

A decision tree

Another example

Splitting criterion for classification

• In each cell, prediction is class with most representation.
• Each observation of other classes is an error.
• Try to create “pure” classes.
• Perfect purity means each cell contains only one class
  \(\Rightarrow\) no errors.

Splitting criterion for regression

• In each cell, prediction is mean.
• Usually try to minimize sum of squared errors.
• Algorithm will try to find splits that separate outliers into their own cells.
• To avoid dependence on outliers,
  • Minimize sum of absolute errors instead, or
  • Choose target variable that does not have outliers

Get some data

``{.p code-line-numbers="3-5|1-8|9|10"}
data = pd.read_sql(
    """
    select ticker, date, ret, roeq, mom12m
    from data
    where date='2021-01'
    """, 
    conn
)
data = data.dropna()
data.info()

<class 'pandas.core.frame.DataFrame'>
Int64Index: 2542 entries, 0 to 2547
Data columns (total 5 columns):
 #   Column  Non-Null Count  Dtype  
---  ------  --------------  -----  
 0   ticker  2542 non-null   object 
 1   date    2542 non-null   object 
 2   ret     2542 non-null   float64
 3   roeq    2542 non-null   float64
 4   mom12m  2542 non-null   float64
dtypes: float64(3), object(2)
memory usage: 119.2+ KB

Fit a classification tree

from sklearn.tree import DecisionTreeClassifier

data['class'] = data.ret.transform(
  lambda x: pd.qcut(x, 3, labels=(0, 1, 2))
)
X = data[["roeq", "mom12m"]]
y = data["class"]

model = DecisionTreeClassifier(
  max_depth=2, 
  random_state=0
)
model.fit(X, y)

View the classification tree

from sklearn.tree import plot_tree
import matplotlib.pyplot as plt

plot_tree(model)
plt.show()

Confusion matrix

from sklearn.metrics import ConfusionMatrixDisplay
ConfusionMatrixDisplay.from_estimator(model, X=X, y=y)
plt.show()

Fit a regression tree

from sklearn.tree import DecisionTreeRegressor

X = data[["roeq", "mom12m"]]
y = data["ret"]

model = DecisionTreeRegressor(max_depth=2)
model.fit(X, y)

View the regression tree

plot_tree(model)
plt.show()

Which are the low ROE, high MOM stocks?

subset = data[
  (data.roeq<=-0.181) & (data.mom12m>1.672)
]
subset

Predicting ranks

data['rnk'] = data.ret.rank(pct=True)

X = data[["roeq", "mom12m"]]
y = data["rnk"]

model = DecisionTreeRegressor(max_depth=2)
model.fit(X, y)

View the regression tree for ranks

plot_tree(model)
plt.show()

Predicting numerical classes

X = data[["roeq", "mom12m"]]
y = data["class"]

model = DecisionTreeRegressor(max_depth=2)
model.fit(X, y)

View the regression tree for classes

plot_tree(model)
plt.show()